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Abstract 

Bohr and Heisenberg suggested that the thermodynamical quantities of 
temperature and energy are complementary in the same way as position and 
momentum in quantum mechanics. Roughly speaking, their idea was that a 
definite temperature can be attributed to a system only if it is submerged in 
a heat bath, in which case energy fluctuations are unavoidable. On the other 
hand, a definite energy can only be assigned to systems in thermal isolation, 
thus excluding the simultaneous determination of its temperature. 

Rosenfeld extended this analogy with quantum mechanics and obtained 
a quantitative uncertainty relation in the form AC/A(1/T) > k where k is 
Boltzmann's constant. The two 'extreme' cases of this relation would then 
characterize this complementarity between isolation {U definite) and contact 
with a heat bath (T definite). Other formulations of the thermodynamical 
uncertainty relations were proposed by Mandelbrot (1956, 1989), Lindhard 
(1986) and Lavenda (1987, 1991). This work, however, has not led to a con- 
sensus in the literature. 

It is shown here that the uncertainty relation for temperature and energy in 
the version of Mandelbrot is indeed exactly analogous to modern formulations 
of the quantum mechanical uncertainty relations. However, his relation holds 
only for the canonical distribution, describing a system in contact with a heat 
bath. There is, therefore, no complementarity between this situation and a 
thermally isolated system. 

1 Introduction 

The uncertainty relations and the principle of complementarity are usually seen 
as hallmaris of quantum mechanics. However, in some writings of Bohr and 
Heisenbergd one can find the idea that there also is a complementary relationship 
in classical physics, in particular between the concepts of energy and temperature. 
Roughly speaking, their argument is that the only way to attribute a definite tem- 
perature to a physical system is by bringing it into thermal contact and equilibrium 
with another very large system acting as a heat bath. In this case, however, the 
system will freely exchange energy with the heat bath, and one is cut ofi^ from the 
possibility of controlling its energy. On the other hand, in order to make sure a 
system has a definite energy, one should isolate it from its environment. But then 
there is no way to determine its temperature. 

This idea is in remarkable analogy to Bohr's famous analysis of complementar- 
ity in quantum mechanics, which is likewise based on a similar mutual exclusion of 
experimental arrangements serving to determine the position and momentum of a 
system. And just as this complementary relationship in quantum mechanics finds 
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its 'symbolic expression', as Bohr puts it, in the uncertainty relation ApAq ^ h, 
one might expect to obtain an analogous uncertainty relation for energy and tem- 
perature, or, perhaps, some functions of these quantities. Dimensional analysis 
already leads to the conjecture that this relation would take the form 

ac/aI > k (1) 

where k denotes the Boltzmann constant. 

However these ideas have not received much recognizition in the literature. An 
obvious objection is that the mathematical structure of quantum theory is radically 
different from that of clasical physical theories. There are no non-commuting ob- 
servables in thermodynamics. Therefore, a derivation of the uncertainty relation 
(H) analogous to that of the uiiual Heisenberg relations is impossible. For example, 
the biography of Bohr by Paiad dismisses his proposal of a complementary principle 
in thermodynamics on the grounds that there does not exist a general uncertainty 
relation like (1) to back it up. 

Nevertheless, several authors^"^'^ have in fact produced derivations of a relation 
of this kind, and defended the idea that it reveals a complementarity between ther- 
mal isolation and embedding in a heat bath. An even more far-reaching claim is put 
forward by Rosenfeld^, Mandelbrot^"^ and Lavenda^"^". These authors argue that 
the uncertainty relation actually blocks any reduction of thermodynamics to a 
microscopic theory picturing an underlying molecular reality, in the same way as 
Heisenberg's relation would forbid a 'hidden variables' reconstruction of quantum 
mechanics. 

However, these claims, and indeed the very validity of (|l|), have been disputed 
by other physicists. An example of this controversy is provided by the polemic 
exchange between Feshbach, Kittel and Mandelbrot in the years 1987-9 in Physics 
Today}^"^^ One of the immediately arising questions is what the exact meaning 
of the A's in (j^) could be. A first thought may be that these uncertainties are 
to be understood as standard deviations of a random quantity, according to one 
of the probability distributions (or ensembles) of statistical mechanics. But in the 
commonly used ensembles to describe a system in contact with a heat bath, the 
canonical ones, temperature is just a fixed parameter and doesn't fluctuate at all. 
On the other hand, in the microcanonical ensemble the energy is fixed! So this 
straightforward interpretation cannot be correct. 

Indeed, all versions of uncertainty relation (Q) which have been proposed in the 
literature employ different theoretical frameworks and give different interpretations 
of the uncertainty A/3, in most cases by using concepts from theories of statistical 
inference. Our purpose is to review and criticize the existing derivations and the 
associated claims (Sections 2 and 3). We shall argue that even though there exist 
valid versions of this relation, it does not express complementarity between energetic 
isolation and thermal contact. 

We also propose (Section 4) an elaboration of the approach of Mandelbrot, 
which is based on the Cramer-Rao inequality from the theory of statistical esti- 
mation. We generalize this approach by means of the concept of distance between 
probability distributions. This statistical distance is a measure of distinguishabil- 
ity: the smaller the distance between two probability distributions, the worse they 
can be distinguished by any method of mathematical statistics. It will be shown 
that this statistical distance leads to an improvement of the Cramer-Rao inequal- 
ity. The resulting uncertainty relation has the virtue of being completely analogous 
to a quantum mechanical formulation. In section 5 we will use this approach to 
investigate intermediate cases between the canonical and microcanonical ensemble. 
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2 The approach from fluctuation theory: Rosen- 
feld and Schlogl 

The best-known proposal for a quantitative uncertainty relation between energy 
and temperature is that of Rosenfeldtl. He obtained the result 

AUAT = kT^ (2) 

for the standard deviations (root mean square fluctuations) of energy and temper- 
ature of a small but macroscopic system in thermal contact with a heat bath kept 
at the fixed temperature Tq. In the case of small fluctuations, AT <C To, A/3 ~ 

and Rosenfeld's result reduces to (|lj). 

Rosenfeld's interpretation of relation (Q) fits seamlessly into the Copenhagen 
tradition. He argues that the meaning of physical quantities has to be obtained 
by operational definitions, i.e. by referring to our experimental abilities to control 
their values. Thus the meaningfulness of quantities depends on the experimental 
context, just like in quantum mechanics. Energetic isolation and thermal contact 
with a heat reservoir are contexts that exclude each other and therefore U and T 
are not simultaneously meaningful. The uncertainty relation symbolizes this 
fact. More generally, Rosenfeld speaks of a complementarity between dynamical 
and thermodynamical modes of description. 

This complementarity, according to Rosenfeld, is equally fundamental as Heisen- 
berg's uncertainty relation. He argues that reference to the underlying microscopic 
constitution of the system will not succeed in restoring a unified description, just 
because the dynamical and thermodynamical concepts are only meaningful in mu- 
tually exclusive experimental conditions. 

For the actual derivation of (^) Rosenfeld refers to the textbook of Landau 
and LifshitzEJ. This book offers a treatment of thermal fiuctuations for a small 
macroscopic system in an environment at fixed temperature Tq and pressure pq. It 
is based on Einstein's postulate, which inverts and reinterprets Boltzmann's famous 
formula S = khiW into: 

W{X) cx e'^-'^-^'/'^' (3) 

in order to assign relative probabilities to thermodynamical states X in terms of 
their entropy S. Here, X denotes a state of the total system (small system and 
environment) and Stot is the total entropy. An equivalent formulation is 

W{X) = M^(Xo)e('^'°'(-^)~^'°'(^''»/''' (4) 

where Xq is the equilibrium state. 

The probabilities W{X) are interpreted as the relative frequencies with which 
the states X occur during a very long time interval, i.e. they describe fluctuations. 
An essential assumption is now that both the small system and the environment 
taken separately are always in thermodynamical equilibrium states. Thus, even 
though they need not be in thermal or mechanical equilibrium with each other, it 
is assumed that they can always be characterized by equilibrium states on which 
the ordinary thermodynamical formalism is applicable. The entropy fluctuation 
5'tot(^) — >S'tot(^o) of the total system can then be calculated in terms of the minimal 
amount of work needed to restore the equilibrium state Xq ■ This can be expressed 
in the state variables of the small system and the fixed equilibrium values po and 
To: 

Stot{X)-Stot{Xo)^-^^H^^ + S{x) (5) 

Jo 

where the variable x denotes the state of the small system. Thus, from (|^) and (||) 
one can determine the fiuctuation probabilities in terms of the state variables of 
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the small system. Choosing a complete set of independent state variables, say x = 
(V,T), and expanding up to second order around its maximum value, one can 
approximate this distribution by a Gaussian probability distribution over V and T. 
Landau and Lifshitz determine the standard deviations of several thermodynamic 
quantities considered as functions of V and T. In particular they obtain (pp. 352, 
356) 

(AT)2 ^ ^ (6) 



and 



Cv 



{AUy^^{To{ — ] -poj kTol — ] +CvkTl (7) 



Here, Cy — (^)y is the specific heat of the system and the quantities in the 
right-hand sides of are all to be evaluated at the equilibrium values. Finally, 
in order to arrive at Rosenfeld simply assumed that the volume of the system 
is constant (so that ^ = 0). 

Let us now see if this result bears out Rosenfeld's interpretation. There are 
several objections. A first group of objections is directed in particular against his 
claim that the result would be 'equally fundamental' as the Heisenberg uncertainty 
relation. First, the relation (||) is obtained by ignoring the first term in the right- 
hand side of (^. This is obviously not a satisfactory general procedure, unless one 
can prove that the deleted term is always non-negative. For usual thermodynamical 
systems, one has indeedtj 



TOO, (|^j^<0 (8) 



so that a more general inequality Af7AT > kT^ results from (^) and (|^). However, 
the inequalities (H) are in turn derived by an appeal to the stability of thermody- 
namical states. The assumption that all equilibrium states occurring in nature are 
stable is obviously not a fundamental law. 

Secondly, the above derivation relies on a Gaussian approximation to (^). This, 
likewise, cannot claim fundamental validity. Moreover, the question how well the 
Gaussian distribution approximates the true distribution (^ depends on the choice 
of the variables to parameterize the macroscopic states. (That is, whether one 
uses the pairs {V,T), {P,V) etc., or prefers T above /3, or perhaps (T — TqY ^ ^ 
temperature scale.) The choice of state variables is usually regarded as a matter of 
convention, and one would not like a fundamental result to depend on this. 

A third objection is that whatever limitations (|^) poses on the simultaneous 
meaningfulncss of the concepts of energy and temperature, a look at the right- 
hand side of this relation immediately suggests that these limitations will become 
negligible if we take the temperature of the heat bath sufficiently low. This again is 
in obvious contrast to Heisenberg's uncertainty relation which provides an absolute 
lower bound for all quantum states. 

Remarkably, it is possible to overcome all the above objections and obtain a 
more generally valid uncertainty relation for the situation considered here. To see 
this, let us choose ([/, V) as the set of independent state variables and write the 
probability distribution resulting from the Einstein postulate in the form 

p(f/, V) = c'e-'5°(^+P«^)+^(^'^)/'= (9) 

and introduce the quantity 
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It is easy to see that /3(C/, V) — {§§-)y, and is thus identical to the usual thernio- 
dynamical definition of the inverse temperature of the system. Using the general 
inequality 

AAAB > |Cov(yl, B)\ := {{A - {A)){B - (B))) (11) 

where (•) denotes the expectation value with respect to and noting that Cov(?7, /3) — 
— 1 (by partial integration and assuming that p{U, V) = when U = 0), one obtains 

AUAP > 1 (12) 

independent of stability arguments, Gaussian approximation or the value of Tq. 
This relation was first obtained by Schlogllj. 

Let us make a few comments on this improved result. Note that there is no 
assumption of an underlying mechanical phase space of the system. Rather, one 
works directly with probability distributions over the macroscopically observable 
variables. Thus, the derivation is not a part of the Gibbsian theory of statisti- 
cal mechanics. Rather, the above treatment, combining orthodox thermodynamics 
with a probability postulate is typical for statistical thermodynamics. In principle, 
it is an open question whether this version of statistical thermodynamics is con- 
sistent with the existence of an underlying microscopic phase space on which all 
thermodynamical quantities can be defined as functions. 

Indeed a remarkable aspect of this version of statistical thermodynamics is that 
quantities like temperature and entropy depend on the probability distribution, via 
the Einstein postulate. This means one cannot vary the probability distribution 
and the temperature independently. Obviously, this aspect alone already provides 
an obstacle to a hidden-variables-style reconstruction. 

The same aspect, however, makes the relation (|l2|) rather different from quan- 
tum mechanical uncertainty relations. It states only that there are non-vanishing 
fluctuations in energy and inverse temperature for the distribution (^. For an 
arbitrary distribution, the quantity /? defined by (^0|) does not coincide with the 
inverse temperature. This limited validity does not license a complementarity in- 
terpretation. For example, for the ideal gas one has (}{U,V) = with Cy a 
constant, i.e. /? is a bijective function of U. Clearly, as Lindhardllil pointed out, any 
claim that 'precise knowledge of U precludes precise knowledge of /3' would be quite 
untenable here. The positive lower bound for AU A/3 is due, in this case, to their 
correlation rather than their complementarity. One is not left, for a given system, 
with a choice of making AU smaller at the expense of A/3 or vice versa. Indeed, the 
relative values of the two uncertainties in (||) are decided by the size of the system 
(which determines the heat capacity Cy), not by context of observation. 

Thus, important objections against Rosenfeld's interpretation of the uncertainty 
relation (||) remain, even in the improved version (|2|). Note also that while Rosen- 
feld speaks of a complementarity between thermal contact and energetic isolation, 
the above treatment nowhere refers to isolated systems. The only experimental 
context considered in the derivation is that of thermal contact with a heat bath. 
Therefore, the basis for Rosenfeld's assertions is very weak. 

Perhaps Rosenfeld's interpret|ation was inspired by a short remark at the end of 
Landau and Lifshitz's discussionllB, where they mention that temperature fluctua- 
tions can also be considered "from another point of view" , and briefly discuss the 
canonical and microcanonical ensembles. They state that one must assume temper- 
ature fluctuations exist also in an isolated system, so that the result (^) "represents 
the accuracy with which the temperature of an isolated body can be specified." 
However, none of this follows from the treatment they actually present. 

Now one could try to provide such an analysis of thermal fluctuations in an 
energetically isolated system along the lines of Landau and Lifshitz. However, this 
will obviously lead to AU = 0, and in the example of the ideal gas, A/3 = as 
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well. The only way in which temperature fluctuations can be obtained is therefore 
either by specializing to systems where (3 is not a function of U alone (e.g. a photon 
gas where j3{U, V) oc (F/C/)^/^), or by generalizing the approach to allow for local 
fluctuations within different parts of the system. However, we shall not pursue this. 

3 Statistical inference 

The approaches to the derivation of thermodynamic uncertainty relations wc; discuss 
next are all related to the fleld of statistical inference. We therefore start this section 
with a short introduction to this subject. The theory of statistical inference has been 
developed in the twenties and thirties by mathematical statisticians, working largely 
outside the physics community. The physicists working in statistical physics have 
for a long time continued thinking about statistics using the concepts laid down in 
the older work of Maxwell, Boltzmann and Gibbs. Still, it has been recognized since 
the sixties that statistical physics can benefit from the ideas and concepts developed 
in mathematical statistics. For our purpose the most significant aspect is that more 
sophisticated concepts of uncertainty are available here than the standard deviation. 
However, the fleld of statistical inference is one in which several approaches exist 
and there is a longstanding debate about its foundations. If statistical physics can 
profit from the developments in statistical inference, it also cannot remain immune 
to this debate, as we shall see. We shall meet four approaches to statistical inference: 
estimation theory, Bayesianism, fiducial probability and likelihood inference. We 
shall see that the so-called Fisher information plays a prominent (but different) role 
in most of them. 

3.1 Estimation theory 

Generally speaking, statistical inference can be described as the problem of decid- 
ing how well a set of outcomes, obtained in independent measurements, fits to a 
proposed probability distribution. If the probability distribution is characterized by 
one or more parameters, this problem is equivalent to inferring the value of the pa- 
ramctcr(s) from the observed measurement outcomes x. Perhaps the simplest, and 
most well-known approach to the problem is the theory of estimation, developed by 
R.A. Fisher. 

In this approach it is assumed that one out of a family {po{x)} of distribution 
functions is the 'true' one; the parameter being unknown. To make inferences 
about the parameter, one constructs estimators, i.e. functions 6{x\, . . . ,x„) of the 
outcomes of n independent repeated measurements. The value of this fimction is 
intended to represent the best guess for 6. Several criteria are imposed on these 
estimators in order to ensure that their values do in fact constitute 'good' estimates 
of the parameter 9. Unbiasedness for instance, i.e. 

/n 
0{xi,...,xn)l[pe{xi)dxi=e (for ah 0) (13) 

i=l 

demands that the expectation of 0, calculated using a given value of 9, reproduces 
that value. If also the standard deviation A^^ of the estimator is as small as possible, 
the estimator is called efficient. 

The so-called Cramer-Rao inequality puts a bound to the efficiency of an arbi- 
trary estimator: 
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where 

is a quantity depending only on the family {pg(x)}, known as the Fisher informa- 
tion. 

The Cramer-Rao inequality ( p^ is valid for any estimator for a given family of 
probability distributions obeying certain regularity conditionsEZl. Specific choices 
of estimators are the maximum likelihood estimators. These are functions 9ml 
which maximize the likelihood i(xi,...,2:„)(^) = Y[7=iP^i^i)- They are asymptot- 
ically efficient, i.e. they approach the bound of the Cramer-Rao inequality in the 
limit n ^ oo. 

Another important criterion in estimation theory is that of sufficiency. Suppose 
that for a given estimator 6{xi, . . . , Xn) the probability distribution function can be 
written as 

Peixi, . . . ,x„) = pe[9)f{xi, . . . ,a;„) (16) 

where po{9) is the marginal distribution of 9 and / is an arbitrary function which 
does not depend on 9. Thus, given the value of 9{xi, . . . ,a;„), the values of the 
data distributed independently of 9. In that case, 9{xi, . . . ,a;„) is 

said to be a sufficient estimator, because it contains all the information about the 
parameter that can be obtained from the data. 

Sufficiency is a natural and appealing demand for estimation problems. Un- 
fortunately sufficient estimators do not always exist. A theorem by Pitman and 
Koopman states that sufficient estimators exist only for the so-called exponential 
family^ i.e. distributions of the form 

pe{x) = exp {A{x) + B{x)C{9) + D{9)) (17) 

where A, . . . , D are arbitrary functions (apart, of course, from the normalization 
constraint). Fortunately, most of the one-parameter distributions that we meet in 
statistical physics do belong to the exponential family. 

Note that the Fisher information for 9 in n observations remains unaffected 
when we restrict the set of data to a sufficient estimator: 

T (a\ [ 1 (dpe{xi,...,Xn)V , , [I ( dpe{9)\ ^ 

''^'^-J Pe{x,.....xA d9 ) 

where pe{9) is the marginal probability distribution of 9. By contrast, Ip decreases 
when the data are restricted to a non-sufficient estimator. In this sense too, a 
sufficient estimator extracts the maximum amount of information about 9 from the 
data. 



3.2 Uncertainty relations from estimation theory: Mandel- 
brot 

Mandelbrot!^ was probably the first to link statistical physics with the theory of 
statistical inference. He obtained a thermodynamic uncertainty relation using the 
methods of estimation theory. 

Like Rosenfeld and Landau and Lifshitz, he adopts the point of view of statisti- 
cal thermodynamics in which the concept of an underlying microscopic phase space 
is superfluous and one works directly with probability distributions over macro- 
scopic variables of the system. In contrast to the previous discussion, where such 
a distribution is obtained from the Einstein postulate, one now starts by assuming 
the existence of a probability distribution P/siU) to describe the energy fluctuations 



7 



of a system in contact with a heat bath at temperature (3. The temperature of 
the system is thus represented by a pararneter in its probabiUstic description. By 
imposing a number of axioms MandelbrotQ is able to determine the form of this 
probabihty distribution. The most important of these is the demand that sufficient 
estimators for (3 should be functions of the energy U alone. This enables him to 
invoke the Pitman-Koopman theorem (|l^) which eventually leads to the form: 

-<^')^^. 

with Z{j3) the partition function, i.e. the—normalization constant and lij{U) the 
so-called structure function of the system.lla 

Mandelbrot considered the question of estimating the unknown parameter (3 of 
this system by measurements of the energy. The Fisher information in this case 
equals 

lF{P)^{^pUf^{U^)p^{U)l. (20) 

Thus, if we apply the Cramer-Rao inequality for unbiased estimators (3, we imme- 
diately find the result: 

A^C/A^/3>1. (21) 

This is Mandelbrot's uncertainty relation between energy and temperature. It ex- 
presses that the efficiency with which temperature can be estimated is bounded 
by the spread in energy. Note that this does not mean that the temperature fluc- 
tuates: it is assumed throughout that the system is adequately described by the 
canonical distribution function ( p^ with fixed j3. Rather, the estimators fluctuate 
(i.e. they are random quantities). Their standard deviation is employed, as usual 
in estimation theory, as a criterion to indicate the quality (efficiency) with which (3 
is estimated. Thus the two A's in the relation ( |2l| ) have different interpretations, 
in contrast to the result of Rosenfeld or Schlogl. 

Let us make some remarks on this result. On first sight, the use of such differ- 
ent interpretations for the two uncertainties in relation ( ^l| ) may seem to reveal a 
striking disanalogy with the quantum mechanical uncertainty relations. However, 
recent work on the quantum mechanical relations has shown that here too it is 
advantageous tp-jemploy concepts from the theory of statistical inference. Already 
several authorsE£rEJ have advocated the Cramer-Rao inequality for a formulation 
of the quantum mechanical uncertainty relations which improves upon Heisenberg's 
inequality. We shall discuss these developments in section ^. Let it suffice to remark 
here that the approach by Mandelbrot is actually in close analogy with these recent 
formulations irujquantum mechanics. 

Mandelbrota calls the spirit of his approach "extremely close to that of the 
conventional (Copenhagen) approach to quantum theory" and argues that the in- 
compatibility of quantum theory and hidden variables is comoaraLle to that of 
statistical thermodynamics and kinetic theory. In later workfflfilld however, he 
no longer claims that statistical thermodynamics is incompatible with, but only 
indifferent to, the use of kinetical or microscopic models. Thus he writes: "Our 
approach. . . realizes a dream of the 19th century 'energeticists': to describe matter- 
in-bulk without jXfiference to atoms. It is a pity that all energeticists have passed 
away long ago."E3. Indeed, his aproach can be readily extended to statistical me- 
chanics by assuming the existence of a mechanical phase space and interpreting the 
distribution (|l9|) as a marginal of a canonical distribution over phase space: 



ZiP) 



H(pi,...,pM\qi,...,qN) = U 

(22) 
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where (pi, . . . ,pN',qi, ■ ■ ■ ,qN) G is the microscopic state of the system and 

H denotes its Hamiltonian. The structure function is then identified with the area 
measure of the energy hyper-surface H(pi, . . . ,pn', 9i, • ■ ■ , <]n) = U. Of course, since 
the evolution of the microscopic state is now dictated by the mechanical equations 
of motion, the validity of the interpretation of the probabilities as time frequencies 
needs additional attention, e.g. by the assumption of an ergodic-like hypothesis. 

Note, however, that such a detailed microscopic description does not help for 
the estimation of (3. The energy is still a sufficient estimator, and therefore contains 
all relevant information about the temperature. No further information is gained 
by specifying other phase functions as well, or indeed the exact microscopic state 
itself. Thus, we can restrict our attention to the distribution over the energy. 

Further, we note that, since Mandelbrot relies on the Cramer-Rao inequality, 
his uncertainty relation is valid only when the canonical distributions are regular 
(cf. footnote Thus, it may fail, for example, for systems capable of undergoing 
phase transitions. 

Mandelbrot's result ( ^l|) applies, like those discussed in section 2, only to systems 
imbedded in a heat bath. Let us now ask whether it can be extended to isolated 
systems in order to see whether there is some kind of complementarity between 
these two contexts. The distribution function is in this case microcanonical 



where e is the fixed energy of the system. 

The first problem one can then raise, in analogy with the previous case, is that 
of inferring the value of the energy parameter e. It is immediately clear however 
that a single measurement of the energy U itself suffices to estimate e with utmost 
accuracy. The Fisher information is infinite, and no informative uncertainty relation 
arises in this case. That is, choosing i — U as estimator for e, we obtain: 



A next and more interesting problem is then what can be said about the temper- 
ature in the microcanonical distribution. This, of course, presupposes-.that one can 
give a definition of temperature of an isolated system. Mandelbrotlij proposes to 
address this problem by regarding the isolated system as if it had been prepared in 
contact with a heat bath, i.e. as if it were drawn from a canonical ensemble. In that 
case the discussion of our previous problem (i.e. of estimating the temperature of 
the heat bath) would have been applicable. His proposal is to treat the assignment 
of a temperature to the isolated system in the same way as the estimation of the 
unknown parameter /3 of this hypothetical canonical distribution. In other words, 
whatever function (3{U) is a 'good' estimator of /3 in the canonical case is also a 
good definition of temperature in the microcanonical case. 

It is interesting, therefore, to consider some specific estimators for f3. Mandelbrot 
mentions three of them. The maximum likelihood estimator Pi{U) is defined as the 
solution of the equation 



p,{U) = S{U-e), 



(23) 



A,e = 0. 



(24) 




(25) 



Other choices are: 



d\nn{U) 
dU 



(26) 



and 



d\nu}{U) 
dU 



(27) 



where 




(28) 
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These functions are in f^ci often proposed as candidate definitions of the tempera- 
ture of isolated systemsO They generaUy yield different values for finite systems, 
but for typical systems in statistical mechanics (consisting of particles with finite 
range interactions), they converge for large numbers of particlesBj For the ideal gas, 
for example, one has /3i(C/) = /32(f/) = 3N/{2U) and PsiU) = {3N-2)/{2U). How- 
ever, in other cases, e.g. for a system consisting of magnetic moments, where uj{U) 
is decreasing in a part of its domain, /32 and /?3 may take opposite signs, even in 
the thermodynamical limit. The problem of choosing a general 'best' temperature 
function still seems to be undecided. 

Even so, following Mandelbrot's proposal, one can associate a proper tempera- 
ture to an isolated system. One might now expect, perhaps from the suggestions of 
Landau and Lifshitz, or from a supposed symmetry in a complementarity relation- 
ship, that there should be unavoidable fluctuations of such a temperature function 
in an isolated system. However, all the candidate functions (3i above are functions 
of U, and thus remain constant in the microcanonical ensemble. Hence one obtains 
(/3i)e = /3i(e) and 

A,U = 0, A,/3, = 0. (29) 

Thus, again, no uncertainty relation is obtained for the microcanonical ensemble. 
This result will clearly hold generally for all candidate temperature functions in 
Mandelbrot's proposal, since the postulate that U should sufficient for (3 implies 
that 'good' (i.e. efficient) estimators depend on U alone. ■— ■ 

This is not the conclusion Mandelbrot draws, however. He proposesll^ to judge 
not only the temperature, but also the uncertainty in temperature from the point 
of view of the canonical ensemble from which the isolated system could have been a 
member. Also the uncertainty in energy is calculated from this counterfactual point 
of view. Thus we simply recover the canonical uncertainty relation ( pT| ) which is 
now, counterfactually, said to apply also to the microcanonical case. But this does 
not seem to be a satisfactory procedure to generalize the validity of a relation. 

We conclude that although Mandelbrot's approach encompasses a discussion of 
both isolated systems as well as systems in a heat bath, the result is still that there 
is no complementarity between canonical and microcanonical ensembles (isolation 
and thermal contact with a heat bath). 



3.3 The Bayesian approach: Lavenda 

Another major school of thought in statistical inference is Bayesianism. According 
to Bayesian statistics, probabilities can be attributed to all kinds of statements, 
in particular also to other probability statements. Thus, one is allowed to assume 
that so-called prior probabilities p{(3) can be attributed to the parameter P in the 
canonical distribution. Furthermore, the canonical distribution is interpreted as a 
conditional probability ppiU) = p{U\P). One is then able, by means of Bayes's 
theorem, 

MU) ^ £~ ,30, 

with Z{U) ^ [ p{U\(5)p{(3)d(5 (31) 



to obtain a so-called posterior probability distribution over [3 conditioned on a given 
value of U . The tenet of the Bayesian approach is that all inferential judgements 
about (3 on the basis of an observed value of U are encapsulated in this posterior 
distribution. In particular, the uncertainty about its value can be quantified by the 
standard deviation of the posterior distribution. 
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The major problem in Bayesian statistics is the choice of the prior distribution 
p. Usually this choice is made with the help of an argument appealing to our prior 
ignorance of the value of j3. Therefore, Bayesian statistics is often regarded as in- 
timately connected to a 'degrees of belief interpretation of probability, in contrast 
to the relative frequency interpretation adopted above. The simplest choice for a 
prior distribution representing ignorance would, of course, be a uniform distribu- 
tion. However, Jeffreysc3 has argued that in the case of a non-negative physical 
quantity /3, appearing as a parameter in a probability distribution, the appropriate 
representation of ignorance is by putting 

p{P) = y^lAF) (32) 

where If{P) is the Fisher information (p^). The main motivation for this choice is 
that this makes the distribution invariant under bijective reparameterization, which 
is obviously desirable for a distribution intended to represent a state of ignorance. 
A drawback is that the prior distribution is often not normalizable. However, 
the posterior distribution usually is. 

Let us now consider the question whether a thermodynamic uncertainty relation 
can be obtained in this approach. That is, we ask whether there is a relation between 
the standard deviation in A[//3 of (|30| ) and the standard deviation A^C/ of 
Unfortunately, in general nothing definite about this question can be said. Indeed, 
it is obvious that both standard deviations contain a parameter, U (the observed 
value of the total energy) and /? respectively, which arc functionally independent. 
As we will see, however, it is possible to derive uncertainty relations for specific 
choices of these parameters. 

Note that since this approach yields a probability distribution over /3, its un- 
certainty can be quantified by means of a proper standard deviation of /?. This is 
possible because the Bayesian approach obliterates, as a matter of principle, the dis- 
tinction between parameters and random quantities. However, this does not mean 
that (3 fluctuates; rather, the posterior distribution from which A/3 is calculated has 
a meaning in terms of degrees of belief, so that A/3 represents a region of epistemic 
uncertainty. j— . 

Using this Bayesian approach, LavendaE~t2l claims to have arrived at an uncer- 
tainty relation: 

AC/ A/3 > 1, (33) 

where the equality sign applies to equilibrium distributions, and a strict inequality 
holds for irreversible processes. He argues that this_result "stands in defense of a 
purely statistical interpretation of thermodynamics''^, just as the Heisenberg uncer- 
tainty principle protects quantum mechanics from a hidden variables interpretation. 
This suggests that the uncertainty relation would forbid a mechanical underpinning 
of statistical thermodynamics. However, he apparently does not wish to go so far, 
because he also writes that the statistical inference approach to thermodynamics is 
analogous to the orthodox approach to quantum theory in the sense that it "circum- 
vents a more fundamental, molecular description." This suggests that a molecular 
description is merely unnecessary, rather than impossible; a position which would 
come close to that of Mandelbrot in his later writings. Howejier, Lavenda seems to 
have changed his views in the opposite direction, since laterEZl he does argue that 
the thermodynamic uncertainty relation excludes a mechanical underpinning: "The 
very fact that uncertainty relations exist in thermodynamics between . . . energy and 
inverse temperature, makes it all but impossible that a probabilistic interpretation 
of thermodynamics would ever be superseded by [a] deterministic one, rooted in the 
dynamics of large assemblies of molecules" . 

We now turn to the derivation of (|3^). Lavenda provides several, but all based on 
diff'erent assumptions. We shall discuss two of his derivations of (|33| ) with equality 
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sign, and one of the corresponding inequality. A major role throughout the approach 
is played by the identification of the value of a maximum likelihood estimator and 
the expected value of a parameter in the posterior probability distribution. He 
is able to trace the assumption back to Gauss's attempts to justify the method 
of least squares, and therefore baptizes it "Gauss' principle". However, he does 
not give a convincing motivation for this assumption. Since in principle estimators 
and expected values are very different quantities, we reject this assumption, and 
have therefore tried to circumvent it as much as possible in our reconstruction of 
Lavenda's work. 

To obtain the uncertainty relation with equality sign, Lavendatl starts from 
the assumption that the system consists of a large number n of non-interacting 
identical subsystems, so that its total energy can be written as Utot = ^i- 
Each subsystem has a canonical distribution. Thus, 

p(C/i,...,C/„|/3) = n|Lgpe-^^- (34) 

and our goal is to consider the posterior distribution 

p(/3|C/i,...,C/„) = . (35) 

We have already noted that the total energy C/tot is a sufficient estimator for /3 in 
the canonical distribution. For the Bayesian approach the cash value of this is that 
the posterior depend on C/tot alone: 

piP\Ui,...,Un)^p{P\Utot). (36) 

In order to obtain the approximate shape of this distribution for large n, one can 
make a second-order Taylor expansion of logp(C/i, . . . , [/„|/3) as a function of f3 
around its maximum value: 

p([/i, . . . , UnW) ^ P0i{Utot)) exp (-^{(3 ~ /3i((7tot))V([/tot)) (37) 



where /3i([/tot) is the maximum likelihood estimate, and 

(9/3) 



J(f/toO - - ( ) . (38) 



/3=/3i((7tot) 



It is easy to show that 



J(C/tot)^ - ( ^'!!f^5^^ ) =/F(/3i(C/tot)). (39) 



l3=nf3i{Utat) 



Thus, this is just another version of the Fisher information. 

For large n one expects that J cx n, so that (|3^) is appreciably different from 
zero only in a small region around Pi(Utot)- Inserting in (|3^), and assuming that 
p{P) behaves reasonably smoothly and does not vanish in this region, one obtains 



J (Utot) /I,. A ,2 



p{(3\Utot) ^ y exp (^--(/3 - /3i)V(C/tot) J (40) 

so that the prior drops out of the posterior. 

The standard deviation of /3 in the Gaussian posterior distribution ( po| ) is obvi- 
ously 
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Also, it is clear from (^) that 

J([/tot) - n (A3^(^^^^)[/)' = (Af/tot)'. (42) 



Conibination of these leads to the uncertainty relation (33) with equality sign, for 
the specific choice (3 = $i{U). 

Lavenda also offers an argument to determine the form of the prior distribution 
from the asymptotic expression and the demand that the expectation value of p 
in the posterior distribution ( ^0| ) should coincide with the maximum likelihood 
estimator for this parameter. Iiiterestingly, this leads to the Jeffreys prior. His 
argument seems to be erroneousJHa but since the prior drops out anyway, this has 
no effect on the remainder of his analysis. 

Lavenda also provides another derivation for the uncertainty relation with equal- 
ity signE'ty Here, no assumption about the number of subsystems and no Gaussian 
approximation are needed. Instead, Gauss' principle is invoked in order to equate 
the expectation value of /3 in the posterior distribution with the parameter in the 
canonical distribution. The entropy S{{U)) is identified with — lnZ((C/)), the sec- 
ond derivative of which equals 

^2=-i\u)f3y. (43) 

Here (U) denotes the expectation value of the energy in the canonical distribution. 
On the other hand, by equating {(3) (jj) with /3, 



a(c/>2 d{u) \ dp ) 



{ApU)-' , (44) 



and the desired result follows. Again the uncertainty relation is derived for a specific 
value for one of the parameters only, but now for U = (U) instead of /3 = /3i. 

Let us now consider relation ( p3| ) with inequality sign. Lavenda notes that such 
an inquality is connected with the Cramer-Rao inequality, but claims that its phys- 
ical content lies in the existence of irreversible processes. His idea is to make use of 
the Second Law, A5i + AS2 > 0, for the entropy increase during a process in an iso- 
lated compound system, and transform this inequality into the desired uncertainty 
relation (i.e. (|3^) with inequality sign) by making use of standard thermodynamic 
relations and certain identifications of thermodynamic quantities with statistical 
ones. 

However, as we shall show, the derivations provided by Lavenda are in error: 
the validity of the uncertainty relation with equality sign is presupposed in the 
statement of the relation with inequality sign. Thus, his claim that the strict 
inequality is instantiated by irreversible processes is mistaken. 

In his booknJ Lavenda considers two subsystems initially at temperatures Ti 
and T2 and assumes they are placed in thermal contact so that they are allowed to 
exchange energy in the form of heat, but not any work. Then, if an infinitesimal 
amount of energy SU is exchanged the total entropy will change by: 

<5^i + 5S2 ^(Jr-^yU> 0. (45) 

Now suppose that both systems are ideal gases, and that as a result of the energy 
exchange, system 1 changes its temperature from Ti to Ti -I- 6T. Then its entropy 
change will be 

5Si = cv In ^^t,'^ ~cv— = -cyTiSP (46) 
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SSi + SS2 = -cvTiSp - 5 = (-cvTiT2^ -A^ (47) 



Hence 



T2 \ SU J T2 

Now, by some inscrutable reasoning, Lavenda argues for the validity of the following 
relations (at least valid up to second order) 

Ip = JjL ^ AU^ ^ (AP)-' = cvT,T2 (48) 

where Ip is the Fisher information. Using these equations one finds 

SSi + 682 = (IfA/3^ - 1) ^ > (49) 

from which the desired inequality follows 

AUA(3 > 1 (50) 

since SU > 0. Thus, according to Lavenda, the uncertainty relation actually stems 
from the Second Law of thermodynamics. 

This argument is obviously erroneous. A first objection is that the validity of 
the left-hand part (|4^) is already confined to the case of reversible proceses only, 
so that the total entropy must remain constant. What is worse, however, is that 
the relations (^Sj) already by themselves imply the stronger relation AC/ A/3 = 1, 
regardless of whether we consider a reversible or irreversible process. 

In conclusion, there is no indication that an uncertainty relation with inequality 
sign can be derived in the Baycsian approach, let alone be explained in terms 
of irreversible processes. Further, we have seen that only one of the derivations 
Lavenda presents for the relation with equality sign, can stand the test of critical 
analysis. This leads to the result that for a system consisting of a large number of 
subsystems, and with a Gaussian approximation for the posterior distribution, the 
relation 

A^^(^)C/Ac/3 = l (51) 

holds asymptotically. 

One cannot help but note the close mathematical connection between this deriva- 
tion and that of Mandelbrot, in spite of their widely different statistical philosophy. 
We have seen that in the large sample approximation, the Bayesian prior drops out 
of equation ( ^ ) and the posterior becomes simply proportional to the likelihood 
function Lj/(/3) = ppiU). On account of the symmetry between (3 and /? in this 
Gaussian, it becomes formally immaterial whether we regard this expression as a 
distribution over (3 (as favoured by Lavenda) or over the estimator /3i, as done 
by Mandelbrot. The standard deviations will in both cases be given by l/^/Tp. 
Thus, Lavenda's result can be seen as a consequence of Mandelbrot's inequality 
(and the fact that the Maximum Likelihood estimator asymptotically saturates the 
Cramer- Rao bound). 

3.4 The approach by fiducial probabihty: Lindhard 

A different approach to thermodynamical uncertainty relations was given by LindharcS. 
Like Rosenfeld, Lindhard restricts his discussion to a system with fixed volume in 
contact with a heat bath. Like Mandelbrot and Lavenda, he uses Gibbsian ensem- 
bles to determine the probability distributions, instead of the Einstein postulate. 
But unlike previous authors, Lindhard considers both the canonical and micro- 
canonical ensembles as well as intermediate cases, describing a small system in 
thermal contact with a heat bath of varying size. 
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The relation he derives is 



{AUf + C^{ATf = kT^C. (52) 

Here, AT and AC/ are standard deviations of temperature and energy of the sys- 
tem, and C = dU / dT is its heat capacity. This relation has a somewhat different 
appearance from the uncertainty relations we have encountered so far, but it is still 
an uncertainty relation in the sense that it expresses that one standard deviation 
can only become small at the expense of the other's increase. 

The relation is intended to cover as extreme cases both the canonical ensemble, 
where, according to Lindhard, AT = and 

(AC/)2 = kT^C, (53) 

and the microcanonical ensemble, for which AU = and 

(Ar)2 = kT^/C. (54) 

Thus, we here have a candidate relation which holds for a class of ensembles, and 
can be seen as expressing a complementarity, not only between temperature and 
energy, but also between the canonical and microcanonical description, in the same 
way as the (improper) eigenstates of position and momentum appear as extreme 
cases in the quantum mechanical uncertainty relations. 

To obtain his result Lindhard starts from the canonical distribution ([l9| ) de- 
scribing a system in contact with an infinitely large heat bath at fixed temperature 
T. The standard deviation of its energy can be expressed as 

(AUf = kT^^^ ^kT^(C). (55) 
This yields (|5^). Next, Lindhard "inverts" the canonical distribution, to obtain 

Puifi) = j%p{U')dU\ (56) 

and interprets this as the probability of the unknown temperature of the heat bath. 
He argues that, due to its infinite size, the heat bath can itself be regarded as an 
isolated system, and can thus be described by means of a microcanonical distribu- 
tion. Hence the inverted distribution ( p6| ) can be interpreted as a microcanonical 
probability distribution for temperature. The standard deviation of this distribu- 



tion should then yield (54). Lindhard does not prove this, but notes that it is 
approximately true when the heat bath is an ideal gas. 

To get to intermediate distributions, Lindhard observes that, just like a canon- 
ically distributed system is in contact with a heat bath of infinite heat capacity, an 
isolated system can be construed as being in contact with a heat bath of zero heat 
capacity. Thus one obtains intermediate cases by considering a heat bath having a 
capacity ^C, with C the capacity of the system itself and < ^ < cxd. Lindhard 
now assumes that the probability distribution for the energy of such a system takes 
the shape of a Gaussian distribution with width (At/)^ = cr^i^/(l + 



the standard deviation in the energy for a canonical distribution (53). Combining 
this result with (jsj) which now reads 



and eliminating ^ we finally obtain the result (|5^). Lindhard also claims, without 
proof, that for more general probability distributions this result holds too, with the 
equality replaced by a greater or equal sign. 
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There are, obviously, a number of objections to Lindhard's approach. On first 
sight, his inversion procedure seems obscure. It is interesting to note, however, 
that the formula ( |5^ ) for a probability distribution over an unknown parameter in 
the light of an observed value U is well-known in another approach to statistical 
inference proposed by Fisher and usually called fiducial probability. This approach 
provides a rival statistical procedure, alternative to both estimation theory and 
Bayesian statistics, and gives Lindhard's inversion technique a theoretical back- 
ground. Fisher restricted this fiducial argument to the cases where a sufficient 
estimator for the parameter exists and where its distribution is monotonous as a 
function oLthe parameter— This approach, however, is controversial and plagued by 
paradoxes.Ej MandelbrottiJ captures general opinion in the remark that the fiducial 
argument is "often regarded as to be carefully avoided" . 

Secondly, there are many gaps in Lindhard's argument. It is not clear that 
the inverted distribution will indeed have the standard deviation ( ^4|) for systems 
other than an ideal gas. Also, his description of the intermediate cases by means of 
Gaussian approximations seems to be too simplified to be persuasive. 

Remarkable is finally that Lindhard's argument involves a shift in the system 
under consideration. The distribution ( p6|) pertains, in first instance, to the heat 
bath. It is used, however, to describe the small system. In order to make this shift 
Lindhard simply assumes that the temperature fluctuations of the total system 
equal those of its subsystems. This is in marked contrast with all other authors on 
the subject. In fact, there are also experimental indications against the validity of 
this assumption£j 



4 Statistical distance 

We now describe a fourth approach to statistical inference, which we favour. This 
is the likelihood approach, which was also developed by Fisher, and later advocated 
by Barnard, Hacking and Edwards. We shall show how this leads to a natural 
measure of inaccuracy in a parameter and use this for the formulation of uncertainty 
relations, both quantum mechanical and for temperature-energy in the canonical 
distribution. 

The idea is here, first of all, that the likelihood function itself conveys all in- 
formation provided by the data about the unknown parameter. In this respect 
the approach agrees with Bayesianism. But now the value of the logarithm of the 
likelihood function is interpreted as the relative support that the data bestow on 
parameter values. Thus, the parameter value for which the likelihood is maximal is 
regarded as the best supported one. This is in close agreement with the use of ML 
estimators in estimation theory. The basic difference with estimation theory is that 
the quality of the inference is judged not by the standard deviation of the estimator 
but by the form of the likelihood function. 

Let us write 

8^(0) Inpeix) (58) 

for the support function and assume for the moment that this is a smooth function 
of 9. A natural way to quantify its width is then by the curvature at its maximum. 
Hence, 

d0^ 



■ logpeix) 



(59) 

gives a measure of the uncertainty in the values of 9 on the basis of observed data 

X. 

Now take a slightly more abstract point of view and consider the inferences to 
be expected if the data x are drawn from a probability distribution pg^. Then, the 



16 



expected support becomes 



S0o{O) ^ - j P0a{x)\ogpe{x)dx (60) 
with a maximum at = 0o- The expected width is then 



dx (61) 

e=ea 



which is just another form of the Fisher information (^_5|). We thus see that in the 
hkehhood approach this expression serves not merely as a theoretical bound for 
the efficiency of all estimators, but has itself a definite statistical interpretation. 
It represents the width of the expected support function, i.e. how easily can be 
distinguished from slightly different parameter values. In this sense it is really a 
measure of how much information one may expect to obtain about the parameter 
from an observation. 

It has been noted by many authors (Rao, Jeffreys) that the Fisher information 
actually defines a metric on the set of all parameter values 9. That is 

5d^]^^W)59 (62) 

provides an distance element for the family {pe\ which is invariant under parameter 
transformations. In fact by allowing multidimensional, or even infinite-dimensional 
parameters, one can extend this metric over all probability distributions on a given 
outcome set. We refer to other works for details.Oo The distance between proba- 
bility distributions is given by 



d{peo,Pe^) = arccos / ^/pejx)pejx^ dx . (63) 



(In the following, we shall write d{9o,9i) for short.) This then provides a measure 
of distinguishability of two probability distributions. It is not only invariant under 
reparametrization, but even completely independent of the original family with 
which we started. Thus it remains useful also when this family is not smooth. This 
use of the Fisher information in the likelihood approach as a metric expressing the 
distinguishability of distributions should not be confused with Jeffreys' proposal to 
use it as a prior probability distribution. 

It is possible to give a lower bound for the right-hand side in terms of the 
endpoints pe^ and pg-^^ only (see the Appendix), to wit: 



d{peoiPei) > arccos 



-1/2- 



(2A6i)2 



(64) 



where a = (1/2) \{9)eo — {9)ei |, and = msx{/S.o^9 , Ae-^9). This relation illustrates 
the fact that when the distance between two probability distributions is small, 
the inefficiency of any estimator is necessarily large. We can put this in a more 
transparent form by defining an inaccuracy in 9, as the smallest parameter difference 
needed to produce a statistical distance greater than a, where a is some convenient 
number between and tt/2. Thus, we define 5a9 as the smallest positive solution 
of 

di9,9 + da9) = a. (65) 

Then the inequality 

6a9^ < (cos~^ a - 1) (66) 
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shows that 5a0 indeed provides a lower bound to estimation efficiency. 

Of course, if the original family is regular (i.e. represented by a smooth curve), 
it is well approximated locally by a geodesic, and the improvement obtained over 
the Cramer-Rao inequality is spurious. By taking 66 = 6i — 6q infinitesimal one 
easily shows from (|6^) that the inequality (^6|) reduces to the Cramer-Rao inequal- 
ity. However, if the family is singular, the Cramer-Rao inequality does not apply. 



whereas the inequality (66) still yields a lower bound to the estimation efficiency. 

An advantage of this approach is that exactly the same approach can andJadeed 
already has been taken to the quantum mechanical uncertainty relations. lj'E3 In- 
deed, consider a set of quantum states \tpx) — e~"^\ip) which are mutually shifted 
in space by a parameter x, and suppose we want to make an inference about this 
parameter. If we perform a measurement of some obervable A with eigenstates |a), 
say, the problem becomes the comparison of probability distributions |(?/'^|a)p for 
the unknown parameter x. Just as in the classical case, we can define a statistical 
distance between two states: 

dyi('0i,V'2) = arccos^ |(-0i|a)(a|V'2)| (67) 

a 

The only important distinction with the classical case is that in quantum theory 
this statistical distance depends on the choice of the observable A. It is natural then 
to introduce the absolute statistical distance as the supremum over all observables: 

c^abs(V'i, "02) = supdA(V'i, V'2) = arccos|('0i|V'2)|- (68) 

A 

If we define an inaccuracy 6aX in the location parameter x just as before, i.e. as 
the smallest value of 6x that solves 

da.hs{'ipx,i>x+5x) = a (69) 



one obtains the relations 



SaxAP > ha (70) 



where AP is the standard deviation in the momentum observable of the quantum 
system. Thus, the inacuracy in the location of the state in space is related to the 
standard deviation in its momentum. 

Similarly, suppose that one wants to estimate the age of a quantum system, i.e. 
to estimate the parameter t in its evolution IV't) = where H denotes the 

Hamiltonian. Then one has 

SatAH > ha. (71) 

For an unstable system, such as a decaying atom, this gives the well-known relation 
between half-life and line width. 

Taking dx (or St) infinitesimal, these relations reduce to the Cramer- Rao version: 



AtfAtH > - 
2 



d{Q) 



dx 
d{f) 



dt 



(72) 



(73) 



for any observable Q (or T) which would be useful for estimaiipg the location (or 
age) of the system. Relation (^2|) was given by Levy-LeblondcS. When applied to 
an 'unbiased' observable, i.e. when 

(Q) ^ i^xlQl^x) ^ X (74) 
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we obtain the standard Heisenberg form of the uncertainty relation, although for 
a more general class of operators than position Q alone. The last-mentioned ver- 
sion of the uncertainty xelation (|7^) for energy and time was already derived by 
Mandelstam and TammtZl 

Let us now apply the concept of statistical distance and the resulting generalized 
uncertainty relations to statistical mechanics. The distance between two canonical 
distributions is 

d{Pl3i , P/32 ) = arccos 

This relation is interesting in its own right. It is a well-known, and oft-repeated 
fact that the canonical partition function Z{i3) is log-convex. This is equivalent 
to the the statement that the above factor between brackets is always less than or 
equal to one. The fact that its arccosine forms a distance function does not seem 
to be so well known. In this case we obtain 

dpAU > a (76) 

with 

^u-j-pj, ^^^^^ 

in analogy with ( [70| ) and ( [ti] ) . Thus the symmetry in these two types of uncertainty 
relations is recovered. The only difference with the quantum mechanical relations is 
that for the canonical family AC/ depends on /3, whereas for a free quantum system 
AP and AT? do not depend on x or t, which explains why an average as in ( [tt] ) is 
unnecessary. 



Z{ 



(75) 



5 Interpolation between canonical and microcanon- 
ical ensembles; the case study by Prosper 

Up till now we have met several unsuccessful arguments aiming to extend the un- 
certainty relation for energy and temperature from the canonical ensemble to the 
microcanonical ensemble. This raises the question whether there actually is a com- 
plementarity between these two ensembles, as envisaged by Bohr or Landau and 
Lifschitz. It is therefore of interest to analyze thermal fluctuations in intermediate 
situations. 

A case study of such thermal fluctuations in a classical ideal gas was given by 
ProsperEa. Consider a system in d spatial dimensions, consisting of N particles in 
contact with a another system of M particles acting as a finite size heat bath. It is 
assumed that both are ideal gases. 

We assume that the total system, consisting of + Af particles, is described by 
a microcanonical distribution with fixed energy e. From this one can calculate the 
probability that the system has an energy U: 

1 fuV-W u\"'-^i 



B{n,m) \ e J \ e J e 
where n = |iV; m = |M ; N,M >1 and 

„/ ^ T(n)T(m) 

B{n,m)^ ' (79) 

i (n -|- m) 

In the limit m —^ oo, ^ = /3oo this distribution tends to the canonical one, with the 
temperature parameter /3oo ■ One the other hand, when the size of the heat bath is 
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negligible compared to the system, i.e. when m becomes very small compared to n, 
the distribution becomes sharply peaked just below the value U — e, resembling the 
microcanonical one. (Note, however, that the representation ( [78[ ) ceases to be valid 
for TO = 0.) Thus, for varying to we have a family of distributions that interpolate 
between the canonical and microcanonical ensembles. 
The standard deviation of the energy is 

{m'= . ^ ^ (80) 

(n + TO + i)(n + TOj^ 

and the question is again what to say about the temperature of the system. Prosper 
uses a Bayesian approach to quantify its uncertainty. However, as we have seen in 
the work of Lavenda, this uncertainty in temperature will not lead to an uncer- 
tainty relation for a finite system. Instead, we shall try Mandelbrot's approach and 
compare this to the statistical distance approach. 

We first assign a temperature-like parameter to the system by reparameterizing 
e. The choice of such a parameter is only straightforward in the limiting case m —^ 
oo. But for finite to the choice is more or less arbitrary. Let us put /3 :— = ""^"^ . 
Thus this parameter coincides with the canonical temperature Poo as to — > cxd. 

Suppose we wish to estimate /3 from a measurement of U . The Fisher information 
in the parameter [3 is: 

oo TO = 1/2, 3/2, 2 

If{P) = { W rn^l (81) 

TO > 2 



n{ra+n- 



{m-2)P 

Using (IsO) we obtain the Cramer-Rao inequality for the parameter P 



{AUf{^Pf> '^^^'~' (to>2). (82) 

[71 + my — 1 

showing the limited efhciency of all unbiased estimators for p. This lower bound is 
independent of P and increasing in to. For to — > oo, it reduces to the canonical value 
1 , already obtained in (|2^) . Thus for the ideal gas we see indeed a gradual transition 
from the canonical case (relation (Ell)) for to — > oo to the microcanonical case 



(relation (29)), where the uncertainty product vanishes. Somewhat disappointing 
is, however, that we cannot carry out a limit to — > 0. Already for to = 2, the 
Fisher information is infinite. This is, however, no indication that the parameter P 
becomes perfectly estimable for small to. Rather, for to < 2 the distributions are 
singular and the Cramer-Rao inequality no longer holds. (Cf. footnote [l7| .) 

Thus the inequality ( ^2[ ) expresses the gradual transition between canonical and 
microcanonical distribution, but not perfectly. 

Here, the method of statistical distance offers a more detailed analysis of the 
situation. The statistical distance for some values of n and to are as follows: 

d{PQ,Pi) = arccos/„i„(6) with b=J^ {Po < Pi) (83) 

y Pi 

and /ii(6) = b (84) 

/3i(5) = (86) 



2 

hi (b) = ^ (-6b + 22b^ + 22b' - 66^ - 3(6^ - 1)4 bg (87) 
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Figure 1: Statistical distance as a function of the relative temperature c^fference 
(3(3) /Po for n = I and various values of m. For m > 2 this distance increases 
linearly for small values of 6(3; for m < 2 the distance element is singular and grows 
more rapidly. 



/5i(fc) - ^(10-562 + 64). (88) 
6 

hn = b" (89) 

Figure 1 shows for the simplest case of n = 1 how these distances behave as a 
function of SP — Pi — Po- It is seen that for small SP the statistical distance grows 
linearly with SP when m > 2, but much faster when m < 2, due to the singularity 
of the associated distributions ([ts]). Thus, with the same choice for the value of a, 
the inacurracy d^P is much smaller for m < 2 than for m > 2. However this does 
not mean that there is no lower bound. For example, for n = m ~ I one finds 

ScPAfjU > (cos-2 a -I) /Vs. (90) 

For larger systems {n > 1), the expression for the statistical distance becomes 
generally more complicated. However, the same trend is observed: if the heat bath 
is small (m < 2), the statistical distance d{P,P + SP) increases more rapidly than 
6P but it behaves regularly (i.e. d{P, P + 6P) oc SP) for m > 2. However there is a 
positive lower bound to the uncertainty product in all cases, viz. 

S^P^pU > """^^ (C:(cosa)-^ - 1) . (91) 

y I L I 1 1 L I J_ 

If we choose a <C 1, the right-hand side is here of order a for m > 2 and for to = 1. 
We recover, therefore, the same conclusion as before: in a gradual transition from 
a canonical to a microcanonical distribution the uncertainty product is bounded by 
a constant gradually approaching zero. We thus see that the uncertainty relations 
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obtained in this approach remain vaUd also for singular families. However, the 
right-hand side is then much lower than in the regular case. 

Of course these conclusions are obtained only for the case study of the ideal gas. 
Yet it remains remarkable that, in the approach using the statistical distance, one 
obtains non- trivial uncertainty relations for regular as well as singular families. A 
serious challenge would then be to apply this to systems capable of phase transitions. 
This, however, falls outside the scope of this paper. 

6 Discussion 

We have reviewed several approaches to the formulation of thermodynamic uncer- 
tainty relations in the existing literature. Only two have a reasonably general deriva- 
tion, free from undesirable simplifying assumptions. The formulation of Schlogl per- 
tains to a version of statistical thermodynamics founded on the Einstein postulate. 
The second formulation is by Mandelbrot. His result is valid for canonical ensem- 
bles both in his own axiomatic version of statistical thermodynamics as well as in 
statistical mechanics. In Schlogl's treatment, the system has a randomly fluctuating 
temperature, in Mandelbrot's case the temperature is identified with a parameter in 
its probability distribution. Its uncertainty is identified with estimation efficiency 
and his uncxatainty relation is valid for regular families, i.e. barring phase transi- 
tions. We have also proposed an extension of Mandelbrot's approach by means of 
the theory of likelihood inference. Here the uncertainty in the temperature parame- 
ter is quantified by means of statistical distance. The uncertainty relation obtained 
in this way is valid also for non-regular families. 

Both Mandelbrot's result and the generalization we proposed are in close analogy 
with formulations of uncertainty relations in quantum mechanics. This provides a 
good reason to take them just as serious as the quantum mechanical relations. 
Still, there are deep differences between statistical mechanics and quantum theory 
in particular in their interpretations of probability. When in quantum mechanics 
a system is described probabilistically, this is usually said to represent the state 
of the system completely; in contrast, the probability distributions in statistical 
mechanics are regarded as 'only' convenient tools. In this case the mechanical phase 
space provides the underlying variables which in quantum mechanics are regarded as 
hidden or even non-existent. The probabilistic description gives only a small part of 
the information concerning the system that could in principle be obtained. But it 
also gives something more. As we have seen, complete knowledge of the microstate 
would not suffice to infer the temperature of the system. What we have added by 
giving a probabilistic description is the notion of the ensemble, of which the system 
of interest is only a member. That is, we have added something which goes beyond 
the particular system we are studying. This points to a common feature between the 
quantum case and the statistical mechanical c;ase. Oncx; we have acxxipted that our 
system is described by some probability distribution out of a certain parameterized 
class, the problem of statistical inference of the parameter occurs in exactly the 
same way, and the uncertainty in this parameter can be quantified with the same 
methods. 

In all the approaches mentioned, there is no uncertainty relation for energy and 
temperature for an energetically isolated system. Therefore, the results do not jus- 
tify a claim for complementarity between energetic isolation and thermal contact, 
as envisaged by Bohr and Heisenberg. Indeed, we conclude that no such comple- 
mentarity exists. Mandelbrot's attempt to give his relation a more general validity 
by regarding the isolated system, counterfactually, as a case which could have been 
described by a canonical ensemble is not convincing. We also come to the conclusion 
that there doesn't exist a complementarity between the microcanonical and canon- 
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ical ensembles. In particular, it is not true (as Lindhard claims) that the extremes 
of any valid uncertainty relation are covered by these ensembles respectively. In 
fact, the study of intermediate cases reveals that the product of the uncertainties in 
energy and temperature gradually changes from zero (microcanonical case) to one 
(canonical case). 

In this respect there are strong disanalogies between the thermodynapair; and 
the quantum mechanical uncertainty relations. It is sometimes arguedd'EI that 
the mathematical basis of the uncertainty relationship in quantum mechanics (the 
Fourier transform between position and momentum eigenstates) is so analogous to 
the Laplace transform relationship between the canonical and microcanonical en- 
sembles, that an analogous relationship should be expected. However, this analogy 
is misleading. The canonical distribution is a convex mixture of microcanonical 
ones, but not vice versa. The grand-canonical distribution, where the number of 
particles is also variable, is in turn a mixture of canonical ones, etc. Thus, these 
ensembles of statistical mechanics are ordered in a hierarchical scale of increasing 
randomness. This is quite different from the symmetry in the Fourier relationship 
between the position and momentum eigenstates of quantum mechanics. 

We finally return to what seems the philosophically most surprising aspect of 
our review. Do thermodynamical uncertainty relations entail an obstacle to a mi- 
croscopic underpinning of thermodynamics, in the same way as their quantum me- 
chanical counterparts forbid the existence of hidden variables? We have seen that 
all of the protagonists in our discussion (except Lindhard) claimed this to be true. 
If so, it would provide an example in classical physics of a situation which is usually 
seen as exclusive to the quantum world and undreamt of in classical physics. 

The first point to make in this connection is that already in quantum mechan- 
ics the uncertainty relations (in the standard form) do not forbid hidden-variables 
reconstructions. A much more formidable obstacle is the theorem of Kochen and 
Specker. Hence, one cannot argue from a supposed analogy here, since the analogy 
fails already in quantum mechanics. The Copenhagen viewpoint that the uncer- 
tainty relations prevent a hidden variables reconstruction of quantum mechanics 
is due to the additional assumption that the physical description by quantum me- 
chanics is already complete. 

This is not to say however, that a mechanical underpinning or 'reduction' of 
thermodynamics to statistical mechanics is straightforward. Indeed, the uncer- 
tainty relations studied here may shed new light on the often heard statement that 
the relation between thermodynamics and statistical mechanics is the archetypal 
example of a successful theory reduction. Only some thermodynamic functions can 
be immediately identified as functions on phase space. It is only for these quantities 
that a complete description of the microstate of the system would be sufficient to 
determine them completely. But for quantities defined statistically, i.e. as param- 
eters or functionals on a probability distribution, like temperature, entropy and 
chemical potential, the complete specification of the microstate will never suffice. 
That is what forms the basis for the uncertainty relations we have studied: no phase 
function can exactly mimick such parameters (for a range of their values). In this 
particular sense, these relations do express the impossibility of a 'hidden- variables' 
style extension of statistical thermodynamics. Observe that no non-commutativity 
is needed for this conclusion. 

Of course this view depends on the definition of temperature (and chemical 
potential etc.) as a parameter in a probability distribution. Another option that 
we have encountered is to consider one of the estimator functions as a definition 
of temperature, rather than as a smart guess (and therefore of course drop the 
qualification 'estimator'). Then temperature becomes a function on phase space, 
irrespective of which probability distribution we choose to describe our system. No 
uncertainty relation for such a temperature function has been established except for 
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the canonical ensembles. But in this case we are still left with the question which 
function to choose, because many can be defined which differ radically for finite 
systems. The reduction of thermodynamics to statistical mechanics therefore still 
seems to be an open problem. 



7 Appendix 

Here we prove the following inequality: for any one-parameter family of probability 
distributions p$ and any unbiased estimator 9 for 9 we have 



/ 



\/ VeAx)peo{x)dx < 1 + 



2A 



-1/2 



(92) 



where A := max.{^0^9, Ag^^)- For n repeated independent observations, this result 
obviously generalizes to 



< 1 



2A. 



-1/2 



(93) 



with A„ := max(Aei^„, Ag^9n). 

The idea behind the proof is to use the Cramer-Rao inequality, not for the curve 
formed by the family pg but for a geodesic between the points p$^ and po2- The 
geodesic between 9i and is the family of distributions of the form 



Pa = {Oiy/Po + Py/plY 

where a varies between and 1, and (3 obeys 

+(3"^ + 2a/3c = 1 



j ^/poPldx. 



(94) 

(95) 
(96) 



Although 9 need not be a good estimator for this geodesic family, the CR inequality 
is nevertheless still valid: 



dot 



A„0 



and we can integrate this along the geodesic. This yields 



d{9o, 9i) = arccos j ^/p^Jxjp^Jx)d2 



da 



l-\/lF{a)da > 



da. 



(97) 



(98) 



Since our purpose is to find a lower bound for the right hand side, we may assume, 
without loss of generality, that (9) a is monotonous in a. Then there exists an 
invertible function y{a) = {0)^- Further, since the integrals are invariant under a 
linear transformation 9 —> cO + d we can arrange that (^)o = —a, (^)i = a. Thus: 



1 I d{S) I 



/ I da 
Jo 2Aa 



-da > 



9 



r d{9) 




Jo 2A9 





f 

J —a 



dy 



(99) 



The strategy is now to find an upper bound for {9-^)a in terms of {9^)o, (6*^)1 
and a. Suppose for the moment that such an upper bound exists, so that, say. 



{0^)a < A\ 



(100) 
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Then we easily obtain 



^ ^ / o = arcsm-. (101) 



or 



Combining this with (|9^ ) we find: 

c^beiiPSa) > arcsin (102) 

J sJpeo{x)peAx)dx < ^1- (^)'. (103) 

It remains to show that an upper bound (100) indeed exists. 

Now the expectation {0'^)a along the geodesic can be written as a convex sum: 

(^2) = a^{§^)o + p^{e')i + 2af3c{§^)2 (104) 

where (.)2 denotes averaging with respect to the auxiliary probability density 



P2ix) = - y/ po{x)pi{x) (105) 

c 

Its maximum value depends on which of the three expectations {0'^)i is the largest. 
We distinguish two cases: (i) {0'^)o or {0^)i is the largest; or (ii) {9'^)2 is the largest 
of the three. 

In case (i) the convex sum reaches its maximum when a is 1 or 0. Clearly the 
argument will be similar in both cases, so let us only consider a — 1. This gives: 

{e^)a<{0^)i^A^ + a^ (106) 



This is the value of A in case (1 ). In serting in (|103| ) leads to the result of theorem 

In case (ii) the convex sum ( |l04[ ) is maximal when = /S^ = 2{i+c) ■ 
find the upper bound 



{e%<^^^^±^ + -^{e% (107) 

1 + c 1 + c 

and we have to find an upper bound for {0'^)2- Using the Cauchy-Schwartz inequality 
and the general inequality {A + x){B + x) < {{A + B)/2 + x)'^ in turn gives: 



{ie-r)i§ + r))2 < -J{{e + rYU{e-rY)^ 

= -^{Al + {a-rf){/\l + {a-rY) 



c 

< -(D^ + ia-rf) (108) 
c 



where :— Ai9, and 



L>2 (Ag + a2)/2. (109) 

This inequality ( |108D is valid for all r; one may choose r so as to optimize the 
strength of the bound. This is the case when r = a/(l + c) and we obtain: 



25 



which is our upper bound for {0'^)2- Combining this with (107) gives the desired 
upper bound for case (ii): 



,1 + 2c 



1 + c 



1 + c 



Inserting this in ( 103 ) yields, after a bit of algebra, 



J Vpeo(x)PsAx)dx < (^1 + 



2 \ -1/2 



(111) 



(112) 



which, in view of (109), is even slightly stronger than the announced theorem. 
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